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Abstract 

Toral introduced so-called cooperative Parrondo games, in which there are 
N > 3 players arranged in a circle. At each turn one player is randomly chosen 
to play. He plays either game A or game B. Game A results in a win or loss of 
one unit based on the toss of a fair coin. Game B results in a win or loss of one 
unit based on the toss of a biased coin, with the amount of the bias depending 
on whether none, one, or two of the player's two nearest neighbors have won 
their most recent games. Game A is fair, so the games are said to exhibit the 
Parrondo effect if game B is losing or fair and the random mixture (1/2)(A + _B) 
is winning. With the parameter space being the unit cube, we investigate the 
region in which the Parrondo effect appears. Explicit formulas can be found 
if 3 < N < 6 and exact computations can be carried out if 7 < N < 19, at 
least. We provide numerical evidence suggesting that the Parrondo region has 
nonzero volume in the limit as N — > oo. 

Keywords: Parrondo's paradox, cooperative Parrondo games, Markov chain, 
stationary distribution, equivalence class, dihedral group, strong law of large 
numbers. 



1 Introduction 

Toral [l j introduced what he called cooperative Parrondo games, in which there are 
N > 3 players labeled from 1 to N and arranged in a circle in clockwise order. At 
each turn, one player is chosen at random to play. Call him player i. He plays either 
game A or game B, depending on the strategy. In game A he tosses a p-coin (i.e., 
p is the probability of heads). In game B, he tosses a po _ coin if his neighbors i — 1 
and i + 1 are both losers, a pi-coin if i — 1 is a loser and i + 1 is a winner, a p2~coin 
if i — 1 is a winner and i + 1 is a loser, and a p3-coin if i — 1 and i + 1 are both 
winners. (Because of the circular arrangement, player is player N and player N + 1 
is player 1.) A player's status as winner or loser depends on the result of his most 
recent game. Of course, the player of either game wins one unit with heads and loses 
one unit with tails. To initialize game B, we could assume that each player tosses a 



fair coin to establish his status as winner or loser; alternatively, we could specify an 
arbitrary initial distribution. But because we are concerned with long-term behavior, 
the initial distribution is usually unimportant. Under these assumptions, the model 
has an integer parameter N and five probability parameters p, po, pi, P2, and p%. 

Toral used computer simulation to show that, with N = 50, 100, or 200, p = 1/2, 
Po = 1, pi = P2 = 4/25, and P3 = 7/10, game A is fair, game B is losing, and the 
random mixture C :— (1/2)(A + B) (toss a fair coin to determine which game to play) 
is winning, providing a new example of Parrondo 's paradox (Harmer and Abbott [2] , 
Abbott [3]). Mihailovic and Rajkovic [4] studied the case N = 3 analytically and 
found that the Parrondo effect is not present for Toral's choice of the probability 
parameters but is present for other choices. They also used analytical methods to 
examine the cases 4 < N < 12 assuming Toral's choice of the probability parameters. 
Xie et al. [5] studied the case N — 4 analytically but their game A differs from Toral's, 
so their conclusions about the Parrondo effect do not apply here. Our interest is in 
the presence of the Parrondo effect for N > 3 and arbitrary choices of the probability 
parameters. We are willing to assume that p = 1/2, so that game A is fair, and that 
Pi = pi as others have assumed, so that the bias of the coin tossed in game B depends 
only on the number of winners among the two nearest neighbors. We also assume 
that the random mixture of game A and game B is the equally weighted one, denoted 
above by C. For fixed N > 3, this still leaves three free probability parameters, po, 
Px, and P3, so our parameter space is the unit cube. 

For 3 < N < 6, explicit formulas can be derived for and /xc, the mean profits 
per turn to the ensemble of N players always playing game B and always playing game 
C, respectively. For 7 < N < 19 (and perhaps slightly larger N), an algorithm can 
be developed that generates exact values for \ib and He, once the parameters po, Pi, 
and ps are specified. By analyzing three choices of the parameter vector (po,Pi,P3) 
(including Toral's), we provide numerical evidence, but not a proof, that \ib and jic 
converge as N — > oo and that the Parrondo effect (i.e., [is < and \ic > 0) persists 
for all N sufficiently large for a set of parameter vectors having nonzero volume. 

Incidentally, there is also the concept of an anti-Parrondo effect (i.e., [is > and 
fj>c < 0); an d we show that there is a symmetry between the two concepts, so that the 
Parrondo region of the parameter space has the same volume as the anti-Parrondo 
region. This should come as no surprise. Indeed, as Harmer and Abbott [2] put it, 
"In a practical sense this is like changing the observer's perspective of the games — 
i.e. whether from the player's or the bank's point of view." 

In addition to the intrinsic appeal of creating a winning game from two fair or 
losing games, Parrondo's games have physical significance. They were originally de- 
vised in 1996 by J. M. R. Parrondo as a pedagogical model of the flashing Brownian 
ratchet of Ajdari and Prost Early work focussed on capital-dependent (Harmer 
and Abbott [7]) and history-dependent (Parrondo, Harmer, and Abbott [8]) games 
for a single player. Multi-player games were introduced by Toral p] [9] , including not 
only the spatially dependent games studied here but also a model in which the role of 
the fair game A is played by a forced transfer of one unit of wealth from one randomly 
chosen player to another, while game B is the original capital- or history-dependent 
one. This redistribution-of-wealth model was studied recently by the present authors 
|10j . That model also motivated a model of Xie et al. [5], in which game B is as 
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in the spatially dependent Parrondo games as above, while game A amounts to a 
forced transfer of one unit of wealth from one randomly chosen player to a randomly 
chosen nearest neighbor. Other multi-player models include the synchronous spatial 
model of Mihailovic and Rajkovic [11] : a two-dimensional spatial model by the same 
authors [12]; a model of Amengual et al. 13 in which win probabilities depend on the 
number of winners (see also Arizmendi [2]); and a model of Wang et al. [15 with 
dependence on capital parity. As the literature on Parrondo's paradox approaches 
200 papers, perhaps the easiest way to get an overview of the subject is to read the 
survey papers [2j [3] cited above. 



2 The Markov chain and its reduction 

The Markov chain formalized by Mihailovic and Rajkovic [4] keeps track of the status 
(loser or winner, or 1) of each of the N > 3 players. Its state space is the product 
space 



N 



£ := {x = (xi, x 2 , • • • , xjy) : Xj € {0, 1} for i = 1, . . . , N} = {0, 1} 

with 2^ states. With the help of some notation, we can specify the one-step transition 
matrix. Let m;(x) := 2xj_i+Xj_|_i, or, in other words, m,i(x) is the integer (0, 1, 2, or 
3) whose binary representation is (x^_i x,_|_i)2. Of course xo := x^r and Xn+i '■= Xi- 
Also, let x % be the element of S equal to x except at the ith component. For example, 
x 1 := (1-xi,x 2 ,x 3 ,...,xat). 

The one-step transition matrix P for this Markov chain depends not only on N 
but on four parameters, po, p\, P2, and p%, which we assume satisfy < p m < 1 for 
m = 0, 1, 2, 3. (This rules out Toral's choice of the probability parameters, at least 
for now, but we will return to this point in Section [5] We do not assume that p\ = P2 
until Section [6j) It has the form 

vi i\ \ N ~ l Pmi(x) if ac* = 0, . 

Pyx, x ) := < _, . * = 1, • • • j N, x e S, (1) 



N 1 q m% ( x ) if ^ = 1, 



and 



P{x,x) := N M ^ 9m«(x) + J! Prrn{m)), xeT,, 
i:xi—0 i:Xi — l ' 



(2) 



where q m := 1 — p m for m = 0, 1,2,3 and empty sums are 0. The Markov chain is 
irreducible and aperiodic. 

The description of the model suggests that its long-term behavior should be in- 
variant under rotation (and, if p\ = P2, reflection) of the N players. In order to 
maximize the value of for which exact computations are feasible, we will use this 
idea to effectively reduce the size of the state space. Our first result describes how 
this is done. We omit the reasonably straightforward proof. 

Lemma 1. Let E be a finite set, fix N > 2, let G be a subgroup of the group of 
permutations of (1,2, ... , N), and let S be a subset of the product space E := E X 
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• • • xE with the property that x := (x%, . . . , xn) G S implies x a := (ar^m, . . . , x a ^)) G 
S for all a G G. Let P be the one-step transition matrix for an irreducible Markov 
chain in S , and let tt be its unique stationary distribution. Assume that P(x„, y a ) = 
P(x, y) for all a G G and x,y g S. Then ir(x a ) — tt(x) for all a £ G and x G S. 

Let us say that x € S is equivalent to y G S ( written x ~ y) if there exists a G G 
such that y — x a , and let us denote the equivalence class containing x by [x\. Then, 
in addition, P induces a one-step transition matrix P for an irreducible Markov chain 
in the quotient set (i.e., the set of equivalence classes) S/~ defined by the formula 

P([x},[y]) := P M)= E P(*,V*), 

y':y'^y a^G:y a distinct 

the second sum extending over only those a G G for which the various y a are distinct. 
Furthermore, if ir is the unique stationary distribution for P, then w is given by 
ir(x) = ff([a;])/|[a;]|, where \[x]\ denotes the cardinality of the equivalence class [x]. 

Remark. The case S — E is of primary interest, but examples in which S is a proper 
subset of E N appear in Section [H] 

The lemma applies to our Markov chain if G is the subgroup of cyclic permutations 
(or rotations) of (1, 2, ... , N), that is, the group generated by 

(a(l),a(2),...,a(N)):=(2,3,...,N,l). (3) 

Indeed, for any cyclic permutation a, 



P(x !T ,(x%) = P(x !T ,(x a ) c 




r -i w (a>„) if (a5a)«r-i(i) = 1 

= \N~ 1 p mt(x ) if x, = 
\^ _1 ?m<(x) ^ %i = 1 
= P(x,x i ) (4) 

for i = 1, . . . , N and all x G S. If p\ = p%, then Q also applies to the order-reversing 
permutation (or reflection) of (1,2,..., N), 

HI), a(2), . . . , a(N)) :=(N,N-1,...,2, 1); (5) 

note that the third equality uses p\ = pi. In this case G is the group generated by 
([3]) and |5| and is known as the dihedral group of order 2N. 

The practical effect of this is that we can reduce the size of the state space (namely, 
2 N ) to what we will call its effective size, which is simply the number of equivalence 
classes. For example, if iV = 3, there are eight states and four equivalence classes, 
namely 

= {000}, 1 = {001,010, 100}, 2 = {011, 101, 110}, 3 = {111}. 

Notice that we label equivalence classes by the number of Is each element has. If 
N = 4, there are 16 states and six equivalence classes, namely 

= {0000}, 
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1 = {0001,0010,0100,1000}, 

2 = {0011,0110,1001,1100}, 
2' = {0101,1010}, 

3 = {0111,1011,1101,1110}, 

4 = {1111}. 

If N — 6, there are 64 states and 14 equivalence classes. A more concise notation in 
this case is 

[0]i/[l] 6 /[3] 6 , [5] 6 , [9] 3 /[7] 6 , [11] 6 , [13] 6 , [21] 2 /[15] 6 , [23] 6 , [27] 3 /[31] 6 /[63]i. 

Here each equivalence class is described by its least element in decimal form. Sub- 
scripts indicate equivalence class sizes. Equivalence classes are ordered first by the 
number of Is each element has in binary form (the / symbol separates different num- 
bers of Is), and second by the least element. If in addition p\ = P2, then, using 
the order-reversing permutation, equivalence classes [11]6 and [13]e coalesce and are 
replaced by [H]i2, so there are only 13 equivalence classes in that case. 

The number of equivalence classes with G being the group of cyclic permutations 
follows the sequence A000031 in the The On-Line Encyclopedia of Integer Sequences 
(http://oeis.org/), described as the number of necklaces with N beads of two 
colors when turning over is not allowed. There is an explicit formula in terms of 
Euler's phi- function. If p\ — p 2 , we can reverse the order of the players, and the 
number of equivalence classes with G being the dihedral group follows the sequence 
A000029 in the OEIS, described as the number of necklaces with N beads of two 
colors when turning over is allowed. Again there is an explicit formula. See Table [T] 

Elements of S are most naturally ordered by regarding them as the binary repre- 
sentations of the integers 0, 1, . . . , 2 N — 1. Elements of E/~ have two natural orderings, 
one of which was described above in connection with the case N = 6. Another ap- 
proach, which is computationally simpler (and adopted in the Appendix), is to order 
equivalence classes simply by the least element. 

To illustrate the one-step transition matrix on the reduced state space, consider 
the case N = 3. The 8x8 one-step transition matrix P is equal to the transpose of 
(9) in Mihailovic and Rajkovic [I], whereas 

f 3q 3 Po \ 

p _ 1 qa Pa + qi + <?2 Pi + P2 

3 qi+q2 Pl+P2 + qz P3 

V 3<z 3 3 P3 / 

Consider also the case N = 4. The 16 x 16 one-step transition matrix P is equal to 
(12) in Xie et al. [5], whereas 
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Table 1: The size and effective size of the state space when there are N players. 



number of 
players 
N 


size of 
state space 
2 jv 


effective size 
not assuming 
Pi = VI 


effective size 
assuming 
Pi = Pi 


3 


8 


1 


4 


i 


1(5 


6 


6 


5 


.32 


8 


8 


6 


64 


14 


13 


V 


128 


20 


18 


8 


256 


36 


.30 


9 


512 


60 


46 


10 


1024 


108 


78 


11 


2048 


188 


126 


12 


4096 


352 


224 


13 


8192 


632 


380 


14 


16384 


1182 


687 


15 


32768 


2192 


1224 


16 


65536 


4116 


2250 


17 


131072 


7712 


4112 


18 


262144 


14602 


7685 


19 


524288 


27596 


14310 


20 


1048576 


52488 


27012 



where rows and columns are labeled by 0, 1, 2, 2', 3, 4, the two suggested methods 
being equivalent. 

More generally, we can give a fairly explicit formula for P. First, define the 
function s : S/~ n- {0, 1, . . . , N} by s([a;]) := x\ + x% + ■ ■ ■ + ijv; it counts the 
number of Is in each element of an equivalence class and is clearly well defined. Then 



P([x],[y}) = { 







(«) + J2i: Xt = l Pm z ( x )) if [y] = [x] 

if a([y]) = *([*])-! 



if = «([*]) + 1 

otherwise 



(8) 



for all [x], [y] € S/~. The first case in Q follows because x 1 / x for all i. Note 
also that, even if s([y}) = s([x}) ± 1, we may have P([x], [y]) = 0; for example, 
P( [001001], [010101]) = 0. We say "fairly explicit" because the evaluation of P still 
requires an enumeration of which can be time consuming. 



3 The stationary distribution 

The unique stationary distribution 7r is too complicated to expect explicit formulas 
except for 3 < N < 6. For N = 3, we can find the unique stationary distribution for 
our eight-state chain by doing the same for a four-state chain. An invariant measure 
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for the four-state chain is (po, 3pi, 3p 2 , P3), where 

Po ■= ffofal +Q2)q-3, Pi -=P0 (<7l +92)g3, P2 -=Po(Pl + Pl)q3, P3 ■= Po(Pl + Pl)P3, 

so an invariant measure for the eight-state chain is (poi Pi, Pi, Pi, pi, Pi, Pi, P3}- The 
stationary distribution follows by dividing each entry by the sum po + 3pi + 3p 2 + P3- 
Incidentally, the N = 3 case is the only case for which the stationary distribution 
is reversible (it in general, and 7r if p\ = p^). A partial explanation is that Q is 
tridiagonal, that is, it corresponds to a birth-and-death chain. 

For N — 4 we can find the unique stationary distribution for our 16-statc chain 
by doing the same for a six-state chain. An invariant measure for the six-state chain 
is (po,4pi,4p 2 ,2p2',4p3,p 4 ), where 

Pa ■= <7o[2<?o<73 + (?i +<72) 2 (<7o + P3)]<?3, 
pi := Po[2q q 3 + (qi + q 2 ) 2 (q + P3)]q3, 

P2 ■= _Po[2<?0<73 + (Pi +P2)(Qi + ?2)(go +P3) + (qi + 92)(l?3 - 9o)]?3 

= po[2p p3 + [pi +p 2 )(qi + q2)(qo+P3) + (pi +p 2 )(qo -P3)]q3, 
Pi' ■= Po[2po93 + (pi +P2) 2 <73 + (qi + q2) 2 po]q3, 

P3 ■= P0[2P0P3 + (Pi +P2) 2 (qo+P3)]q3, 

Pi ■= Po[2poP3 + (pi +p 2 ) 2 (qo+P3)]P3; 

here we have given two formulas for p 2 so that we can see at a glance that it is 
positive. Thus, an invariant measure for the 16-state chain is (po, pi, pi, p 2 , Pi, p2', 
Pi , P3 , Pi , P2 , Pi' , P3, pi, P3, p3, Pi) ■ The stationary distribution follows by dividing 
each entry by the sum p + Api + Ap 2 + 2p 2 ' + ^P3 + Pi- 

In a similar way, we have also found formulas for the unique stationary distribution 
in the cases N = 5 and N = 6 (assuming p± = p 2 in the latter case), but they are 
considerably more complicated and consequently will not be given here. In particular, 
we have not shown algebraically, as we have for iV = 3 and N = 4, that each term is 
positive; for that we must rely on Markov chain theory. 

Notice that, when N = 3 or N = 4, the stationary distribution depends on pi and 
p 2 only through p\ +p 2 . This property also holds when N = 5 but fails when N > 6. 
When it holds, it implies that there is no loss of generality in assuming p\ = p 2 . The 
reason this property holds when N = 3 or N = 4 is that P in Q and ^ depends on 
Pi and p 2 only through p\ + p 2 ■ The same is true when N — 5 but not when N = 6 
because, for example, P([001011], [011011]) = (l/6)pi. 

4 Strong law of large numbers 

Is there a strong law of large numbers (SLLN) and a central limit theorem for the 
sequence of profits by the ensemble of N players? More specifically, does Theorem 1 
of Ethier and Lee [TB] apply in this context? Let us recall the statement of that 
theorem. 

Consider an irreducible aperiodic Markov chain {A„}„>o with finite state space 
Sq- It evolves according to the one-step transition matrix P = (Pij)i,j£E - Let us 
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denote its unique stationary distribution by the row vector 77 = (wi)i£-£ . Let w : 
So x Eo i— > R be an arbitrary function, which we write as a matrix W = (w(i,j))i t j^s 
and refer to as the payoff matrix. Define the sequences {£ n }n>i and {S n } n >i by 

£ n :=w(X n - U X n ), n>l, (9) 

and 

S n :=& + ••• + £„, n>l. (10) 

Let II denote the square matrix each of whose rows is 7r, and let Z := (J— (P — II)) -1 
denote the fundamental matrix. Denote by P and P the Hadamard (entrywise) 
products P oW and P o W o W (so Py := Pijw(i,j) and Py := Pijw(i, j) 2 ). Let 
1 := (1, 1, . . . , 1) T and define 

^:=ttP1 and a 2 := ttPI - (ttPI) 2 + 2ttP(Z - n)Pl. (11) 

Theorem 2 (Ethier and Lee [16] )■ Under the above assumptions, and with the dis- 
tribution of Xq arbitrary, lim n _>. 00 n _1 E[S' n ] = (J,, 

> /j, a.s., 



limn^oon 1 Var(S' n ) = a 2 , and, if a 2 > 0. 

S„ — nji 



■4 N(0,1). 



If li = and a 2 > 0, then — oo = liminfn^oo S n < limsup^^^ S n = oo a.s. 

Remark. The abbreviation "a.s." stands for "almost surely," meaning "with proba- 
bility 1." The symbol — >d denotes convergence in distribution. A game is winning if 
/i > (hence lim5„ = oo a.s.), losing if /i < (hence limS",! = — oo a.s.), and fair if 
/i = (hence — oo = liminf S n < limsup5„ = oo a.s., assuming a 2 > 0). 

It appears at first glance that the theorem does not apply in the present context 
because the payoffs are not completely specified by the one-step transitions of the 
Markov chain. Specifically, a transition from a state x to itself results whenever a 
loser loses or a winner wins, and the transition probability is given by ([2]). And 
yet Q and (10 1 suggest that w(x,x) should be ±1. Does the theorem need to be 



generalized so that w(i,j) is not simply the payoff when X n -\ = i and X n = j but 
rather is the conditional expected payoff given A„_i = i and X n — j? Actually, it 
is more convenient to leave the theorem as it is and instead generalize the Markov 
chain. 

Our original Markov chain has state space S := {0, 1}^ and its one-step transition 
matrix P is given by ([I]) and ([2]). Let tt denote its unique stationary distribution. 
We augment the state space, letting E* := E X {1,2,..., N} and keeping track not 
only of the status of each player as described by x € E but also of the label of the 
next player to play, say i. The new one-step transition matrix P* has the form 



N 1 p nH ( x ) if Xi = 0, 
N~ 1 q mi ( x ) if Xi = 1, 



P*((x,z),(x\j)):={" ^ (*,»)€£*, j = l,2,...,A, 
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and 

P*((x,i),(x,j)) := {"~_\ qmi{x) [i f Xi = °' (x,i) £ S*, j = 1,2, . . . , N, 

where q m := 1 — p m for m = 0, 1, 2, 3. This remains an irreducible aperiodic Markov 
chain, and its unique stationary distribution 7r* is given by ir*{x,i) = N~ 1 it{x). 
Further, the payoff matrix now has each nonzero entry equal to ±1, so the theorem 
applies. 



Therefore, by (11), the mean parameter in the SLLN has the form 

N 

V = 7T*-P*1 = 7r ( :E )X! N ~ 1 \Pm i (x) -?m,w]- 

Alternatively, this can be rewritten as 

H = ttPI = frPl, (12) 

where 1 is the column vector of Is of the appropriate dimension and P and P have 
new meanings. Specifically, P is obtained from P, and P from P, by replacing each 
Qm by — q m . This "rule of thumb" requires some caution: It must be applied before 
any simplifications are made using q rn = l~p m . For example, the rule applies directly 
to ([6| but not to Q because the Is in the (1, 1) and (3,3) entries are po + qo and 
Pi + 93, respectively, and the 2 in the (2, 2) entry is p\ + qi + P2 + <Z2 • (Recall that 
rows and columns are labeled by 0, 1, 2, 2', 3, 4.) 

The main consequence of Theorem 1 from our perspective is the SLLN it ensures. 
Let 5*^, , and S% denote the cumulative profit after n turns to the ensemble of 
players playing game A, B, and C :— "fA + (1 — j)B. Then 

cA cB cC 

!>/x J 4 = 2p— 1 a.s., > fiB a-s., > He a -S- 

n n n 

Of course, there is also a central limit theorem, but we will not try to evaluate its 
variance parameter. 

We conclude this section with an application of the SLLN. Let us denote [i of ( fl2| ) 
by [i{j>§,p\,pi,pz) to emphasize its dependence on the probability parameters. (We 
do not assume that p\ — p2-) 

Corollary 3. With q m := 1 — p m for m = 0,1, 2, 3, we have 

M(Po,Pi,P2,P3) = -M?3, 92,9i, qo). 

In particular, if po + P3 = 1 and pi + P2 = 1, then ^{po,Pi,P2,Pz) = 0. 

Proof. We prove this via a coupling argument. Define the function rj : S i— > £ by 
rj(x) := (1 — x\, 1 — X2, . . . , 1 — xn)- Let {X(n)} be the Markov chain in E with 
initial state x and one-step transition matrix pPo>pi,P3,pa given by ([!]) and ([2J), where 
the superscripts are merely intended to emphasize the probability parameters. Then 
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X'(n) := rj(X(n)) defines a Markov chain {X'(n)} in E with initial state x' :— r/(x) 
and one-step transition matrix pi3><a<ii,qo _ ^he ^ w0 processes are coupled, using the 
same sequence of players and the same sequence of coin tosses. 

To help clarify this, let us consider the special case TV = 3, in which there are 
three players 1, 2, and 3 whose collective status is described by {X(n)} and three 
players 1', 2', and 3' whose collective status is described by {X'(n)}. Suppose X(0) = 
x := (0,1,0) and X'(0) = x' := rj(x) = (1,0,1). If player 2 is chosen to play (and 
determine X(l)), then player 2' will also be chosen to play (and determine X'(l)). 
Player 2 is required to toss a po-coin because = (0 0)2 and the first probability 
parameter of {X(n)} ispo, and player 2' is required to toss a go- com because 3 = (1 1)2 
and the fourth probability parameter of {X'(n)} is qo. We can use the same coin 
toss for both players, except that if player 2 sees heads, then player 2' sees tails, 
and vice versa. So a loss by player 2 (equivalently, a win by player 2') results in 
X(l) = (0, 0, 0) and X'(l) — (1, 1, 1), whereas a win by player 2 (equivalently, a loss 
by player 2') results in X(l) = x and X'(l) = x' . The coupling proceeds in this 
manner at each turn. 

Let S^ 0,Pl ' P2 ' P3 be the cumulative profit to the ensemble of N players after n 
turns, where again the superscripts emphasize the probability parameters. Then, by 
the SLLN, 

M(Po,Pi,Pa,J*) = lim rC l S******* 

71— 1-OQ 

= - lim n^S^^" =-M<?3,<Z2,?i,9o) a.s., 

n— >oo 

where the middle equality holds because each win in the {X(n)} process corresponds 
to a loss in the {X'(n)} process and vice versa. □ 



5 Reducible cases 

We have assumed that < p m < 1 for m — 0, 1, 2, 3, which ensures that our Markov 
chain is irreducible and aperiodic. Can we weaken this assumption? Let us consider 
several cases. We denote by € E the state consisting of all 0s, and by 1 € E the 
state consisting of all Is. 

1. Suppose po = 1 and < p m < 1 for m = 1, 2, 3, as Toral [T] originally assumed. 
Then state cannot be reached from E — {0} and P, with row and column 
deleted, is a stochastic matrix that is irreducible and aperiodic. Note that Lemma 1 
is applicable with S :— E — {0}. Theorem 1 also applies with E := S, or we could 
take Eo :— E; it does not matter because P will have a unique stationary distribution 
7r, which necessarily satisfies 7r(0) = 0. The reason we might prefer Eo := E is that 
any formula obtained assuming < p m < 1 for m — 0, 1,2,3 will remain valid after 
substituting po = 1. 

2. Suppose po = and < p m < 1 for m = 1,2,3. Then state is absorbing, 
and absorption eventually occurs with probability 1. Hence S„ — S^_ 1 = — 1 for all 
n sufficiently large, so /is = —1, as noticed by Xie et al. [5]. 

3. Suppose ps — and < p m < 1 for m = 0, 1, 2. This is analogous to case 1. 
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4. Suppose ps = 1 and < p m < 1 for m = 0, 1, 2. This is analogous to case 2, 
except /is = 1. 

5. Suppose po = 1) P3 = 0, and < p m < 1 for m — 1,2. Then states and 
1 cannot be reached from E — {0, 1} and P, with rows and 1 and columns and 
1 deleted, is a stochastic matrix. However, it is not irreducible (unless N = 3), so 
Lemma 1 does not apply with S := £ — {0, 1}. If N is even, then the two states 
0101 • • • 01 and 1010 • • • 10 in which 0s and Is alternate are absorbing, and from either 
state there is a win of one unit with probability 1/2 and a loss of one unit with 
probability 1/2. Consequently, fig = 0, regardless of pi and p 2 . 

6. Suppose po — 0, ps = 1, and < p m < 1 for m = 1, 2. Then both and 1 are 
absorbing, and absorption occurs with probability 1. The probability of absorption at 
1 depends on the initial state (or equivalence class), and can be calculated for small 
N > 3. For example, in the case N = 4 with p\ = p 2 , we can derive formulas for \xb 
as a function of pi, depending on the equivalence class of the initial state, and the 
results are consistent with the simulations of Xie et al. [5]; compare their Fig. II. 1 (p. 
410). The details are left to the reader. 



6 The Parrondo region 



If we denote /z of (12) by p-(po,Pi,P2,P3) to emphasize its dependence on the prob- 
ability parameters, then the mean profits per turn for game A, game B, and game 
C := jA + (1 - j)B are p, A ■= v(p,P,P,p) = 2p - 1, p, B := (i(po,Pi,P2,P3), and 

:= fi(ro, r\, r2, 7*3), where r m := 7p + (1 — j)p m f° r m = 0,1,2,3. The Parrondo 
effect is said to be present if pa < 0, [is < 0, and nc > 0. The anti- Parrondo effect 
is said to be present if > 0, ps > 0, and [ic < 0. 

In what follows we assume for convenience that p = 1/2 (game A is fair), p\ = 
Pi (the bias of the coin tossed in game B depends only on the number of winners 
among the two nearest neighbors), and 7 = 1/2 (the random mixture of games A 
and B is the equally weighted one). In particular, for fixed N > 3, we have three 
free probability parameters, po, pi, and p$, so our parameter space is the unit cube 
(0, l) 3 := (0,1) x (0,1) x (0,1). With caution (see Section [5}, we can also include 
parts of the boundary. Of interest are the regions in the parameter space in which 
the Parrondo effect (i.e., /is < and pc > 0) appears and the anti-Parrondo effect 
(i.e., /is > and fic < 0) appears; let us refer to them as the Parrondo region and 
anti-Parrondo region. 

Theorem 4. Fix N > 3 and assume as above that p = 1/2 in game A, pi = p 2 
in games B and C, and 7 = 1/2 in game C. With q m := 1 — p m for m — 0, 1,3, 
the parameter vector (po,Pi,P3) belongs to the Parrondo region if and only if the 
parameter vector (93,91,90) belongs to the anti-Parrondo region. In particular, the 
Parrondo region and the anti-Parrondo region have the same volume. 

Proof. Let \ib and fic denote the means for the parameter vector (po,Pi,Pi,P3), and 
let /i* B and /i c denote the means for the parameter vector (93, 91, q±, qo). Then, by 
Corollary 1 (and using p x = P2), Ms = n(Po,Pi,Pi,P3) = ~K<l3, <h, ?i> ?o) = ~Mb 
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and Mc = M((l/2+Po)/2 s (l/2 + Pl )/2,(l/2 + pi)/2,(l/2+p3)/2) = -A»((l/2 + 
g 3 )/2, (l/2 + gi)/2, (l/2 + <&)/2, (l/2 + g )/2) = Therefore, ^ B < and /i c > 

if and only if /i* B > and n* c < 0. 

For the second conclusion, we define the mapping A : (0, l) 3 H- (0, l) 3 by A(p ,pi, 
P3) — (1 ~P3i 1 1 — Po)- This one-to-one transformation has Jacobian identically 
equal to 1, so it is measure preserving. Since it maps the Parrondo region onto the 
anti-Parrondo region, the two regions must have the same volume. □ 

It will therefore suffice to focus our attention in what follows on the Parrondo 
region. 



6.1 N = 3 

Using the stationary distribution derived above and the mean formula (12 1, together 
with the assumption that p\ — P2, we find that 

Pi(po + 93) - 93 n „s 

M-B = -7. ; • (13) 

PoPx + 2p q 3 + ft g 3 



Since p = 1/2 in game A and 7 = 1/2 in game C, by (13) with p m replaced by 
(1/2 + p m )/2 for m = 0, 1, 3, we have 

2pi (1 + Pa + 93) - <7o - 3g 3 



1 + 3p + 2p Pi + 4p <?3 + 2pip 3 + 2(?i + 5g 3 



Therefore, the Parrondo region is described by pi(po + Q3) — Q3 < and 2pi(l +po + 
93) — 5o — 3(73 > or, equivalently, 

90 + 3(7 3 Q-3 

< Pi < , ■ (14) 



2(l+po + (73) Po + 93 - 

The first inequality is equivalent to (24) in Mihailovic and Rajkovic [4 . There exists 
such a pi if and only if min(/Jo, Qo) < P3 < m a x (Po> Qo)- I n particular, the area of the 
region in the (po,P3) unit square for which there exists a p\ satisfying ( |14[ ) is equal 
to 1/2. 

With the parameter space being the {po,P3,Pi) unit cube, the Parrondo region is 
the union of two connected components. See Figure [l] Two straightforward iterated 
integrals yield its exact volume, (91n9-81n8 - 3)/8 w 0.0174361. 

Toral's [T] games were inspired by the one-player history-dependent games intro- 
duced by Parrondo, Harmer, and Abbott [S]; in their game B, the player tosses a 
Po-coin, a pi-coin, a p2-coin, or a p3-coin if his two previous results are loss-loss, loss- 
win, win-loss, or win-win, respectively, with the second result being the more recent 
one. There is an interesting relationship between Toral's three-player game B and 
the one-player history-dependent game B. 

For i — 1,2,3, let Pi denote the 8x8 one-step transition matrix for the Markov 
chain in £ with N — 3 corresponding to player i being chosen to play at each turn 
(as usual, the coin tossed depends on the status of the nearest neighbors). Then 
the one-step transition matrix P for Toral's game B with TV = 3 can be defined as 
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Figure 1: When N — 3 (upper left), N = 4 (upper right), N = 5 (lower left), or 
N = 6 (lower right), the green (or light) surface is the surface [1b = 0, and the 
red (or dark) surface is the surface jj,c — 0, both in the (po,P3,Pi) unit cube. The 
Parrondo region is the region on or below the green surface and above the red surface, 
while the anti-Parrondo region is the region on or above the green surface and below 
the red surface. (Assumptions: p = 1/2 in game A, pi = p 2 in games B and C, and 
7 = 1/2 in game C.) 
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P := (1/3) (Pi + P2 + P3). It can be shown that the stationary distribution of P 
coincides with the stationary distribution of P1P2P3 when p\ — P2- The one-step 
transition matrix P1P2P3 corresponds to the game in which the three players 1, 2, 
and 3 toss coins in the order named, and this game can be coupled with the one- 
player history-dependent game, using the same sequence of coin tosses, because the 
two most recent players are also the two nearest neighbors. 



The result is that the mean profit fis in (13) is the same as the mean profit per 
turn for the one-player history-dependent Parrondo game B, assuming p\ — P2- This 
then implies that the Parrondo regions are also the same, that is, the Parrondo region 



for the one-player history-dependent games is given by ( 14 ) . 



6.2 N = A 

Again we have an explicit formula for (is (algebraically equivalent to that of Xie 
et al. [5]), from which we can derive a similar formula for /ic- Specifically, /is = 
li(po,Pi,Pi,Pa), where 

/ s /o(PO,P3) + 4(l+Po)(?0+P3)?3Pl-2(go+P3)(?0-P3)Pl 

V{P0,Pl,Pl,P3) ■= j x 

SfoCPO.Pl.P3j 

with / (po,P3) : = -(3- 2p 3 ~ 3po + 2 PoP3 ~P% + tyoPa ~ 2popi) and go(Po,Pi,P3) ■= 
(3 + 6p Q - 2p 3 - 3p 2 - 2 PoP3 -pl + 12p 2 oP3 - 4 Po p 2 3 - 8p 2 oP 2 3 ) - 4( 9o + p 3 + 2p 2 + 
2p p 3 )q 3 p 1 +2{l+Ap Q -p 2 ~2p Q p 3 -p 2 )p 2 , and M c = m((V 2 +Po)A (l/2+ Pl )/2, (1/2+ 
Pi)/2, (l/2+p 3 )/2). The condition for the Parrondo effect amounts to two quadratic 
inequalities in pi with polynomial coefficients in p and p 3 . Solving these inequalities 
and assuming < p m < 1 for m = 0,1,3, we find that the Parrondo region is 
described by po + p 3 < 1 , 

i < (1 +Po)g 3 - y/ (i +Po) 2 qj + (go -P3)f{Po,P3) 



qo -P3 



if (1 + p ) 2 q 2 + (q - P3)f(Po,P3) > 0, where f(p ,p 3 ) := [p (3po - 2p 3 - 2p Q p 3 + 
2p§) - {3+p 3 )q 3 }/{2(q +p 3 )}, and 



Pi > 



](Po,P3) - \/g(pa,P3) 2 + 4(q -P3)h{p ,p 3 ) 
4(<7o - P3) 



where g(p Q ,p 3 ) := 13 + 8p ~ 8p 3 ~ 4p p 3 and h(p ,p 3 ) := (-48+ 14p + 30p 3 + 13pg - 
8poP3 + 3p 3 — 4poP3 + 4popf)/(l + qo + p 3 )- We note that pi > 1/2 in the Parrondo 
region. See Figure [T] The region is connected, and a numerical integration yields 
0.0293350 as its approximate volume. 

6.3 iV = 5,6 

When N = 5 or iV = 6, we have explicit, albeit very complicated, formulas for /j,b 
from which we can derive similar formulas for /ic; in the case N = 6, our formula 
assumes that p\ — P2 for simplicity. It would be impractical and uninformative to 
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state those formulas here. Instead, see Figure [T] which shows that the Parrondo 
regions in the two cases are surprisingly similar. In both cases they appear to be the 
union of three connected components. 

There is a subtle technical issue here. We can solve the quadratic inequalities in 
the N = 4 case, but the quartic and octic inequalities in the N = 5 and N = 6 cases 
are less tractable. We expect that \lq < below the surface \ib = and nc > 
above the surface fic — 0, but we do not have a proof. 



6.4 7<iV<19 

When N > 7, wc no longer have explicit formulas for fiB but we can in principle 
compute it exactly for arbitrary values of the probability parameters by enumerating 
the state space E/~ (i.e., the set of all equivalence classes) and calculating the one- 
step transition matrix P and the related column vector PI as functions of p m and 
q m (as if p m and q m were unrelated) for m = 0, 1, 2, 3. We then specify the desired 
numerical values of the probability parameters p m (and set q m := 1— p m ) and evaluate 



the unique stationary distribution 7r. Finally, we use the rightmost expression in ( 12 ) 



for hb- The advantage of this approach is that, once P and PI are found, we 



can apply ( 12 ) for arbitrary choices of the probability parameters without the time- 
consuming re-cnumeration of the equivalence classes. 

Computations were done on a MacBook Air with 2 GB of RAM using Mathematica 
8. The Mathematica program we used is displayed in the Appendix for N = 10; it is 
also available at http://www.math.utah.edu/~ethier/program.txt or http://yu. 
ac .kr/~leejy/progra m.txt[ For N > 17, we needed more memory. Computations 
were done on an IBM System x3850 X5 with 1 TB of RAM using a Linux version of 
Mathematica 8. The runtime for the case N = 20 was estimated at over seven weeks, 
so we stopped with N = 19. 



6.5 Comparison 

Table [2] summarizes our estimates of the volume of the Parrondo region. For 3 < 
N < 13 we used two methods for estimating this volume. The first, which might be 
called the Riemann sum approximation, consists of evaluating \±b and fic at each of 
the points (2i + l,2j + l,2k+ l)/200 for i,j,k = 0,1,..., 99, and determining the 
proportion of such points at which the Parrondo effect is present. (This is just the 
Riemann sum for the indicator function of the Parrondo region.) The second method 
is by simulation. Our simulation estimate is also based on 10 6 points but they are 
chosen randomly from the uniform distribution over the unit cube. The Riemann 
sum approximation has the advantage of being repeatable, whereas the simulation 
estimate allows estimation of the error in the approximation. 

The volume of the Parrondo region appears to be getting smaller as N increases. 
We expect that it will converge to a nonzero limit, based not on Table[2]but on Tables 
[3] and [4] as we now explain. 

Mihailovic and Rajkovic [4] studied the presence of the Parrondo effect in the case 
of Toral's choice of the probability parameters, namely p — 1/2, p Q = 1, pi = P2 = 
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Figure 2: When N = 7 (left) or N = 8 (right), the Parrondo region is sketched by 
plotting points that belong to it. Two million points were tested for each figure, 
one million equally spaced points and one million randomly chosen points. Because 
this figure lacks the three-dimensionality of Figure [I] we use color (or shading) as a 
partial substitute. The point at (po,P3,Pi) is colored using the Mathematica function 
Hue[pi]. As h varies from to 1, the color corresponding to Hue[/i] runs through red, 
yellow, green, cyan, blue, magenta, and back to red again. See the color scale on the 
right. (Assumptions: See the caption to Figure [lj) 

4/25, and P3 = 7/10 (actually, they took p = 0.499). They obtained estimates of 
Plb and fie using exact computations for N = 3, unspecified analytic methods for 
4 < N < 12, and simulation for N > 12. Table [3] confirms their findings using exact 
computations of /z B and fi c . For example, p, B = -599823882743/31695346763173 
when N — 6, which we round to six significant digits in Table [3] Notice that /ic 
seems to have stabilized to six significant digits by N = 11, so it appears that all 
of the variation in [ic in the upper graph of Figure 2 of [4] is due to experimental 
error. However, [i B is more oscillatory. It seems to have stabilized to three significant 
digits by N = 19, so it appears that most of the variation in \i B in the lower graph 
of the same figure is due to experimental error. This explains why we have not used 
simulation to estimate fi B and /ic for N > 20. 

Table [4] analyzes two other second point on the boundary of the unit cube 

and a point in the interior. In both cases \x B seems to have stabilized more quickly 
than in the case of Toral's choice of the probability parameters. 

7 Conclusions 

We considered the spatially dependent Parrondo games of Toral pQ, which assume 
N > 3 players arranged in a circle, and in which the win probability for a player 
depends on the status of the player's two nearest neighbors. There are three games, 
game A without spatial dependence, game B with spatial dependence, and the ran- 
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Table 2: Estimated volume of the Parrondo region. The integrated volume is obtained 
by integration or numerical integration and is rounded to six significant digits. For 
the Riemann sum approximation, the unit cube is divided into (100) 3 cubes of equal 
size, then the estimate is the proportion of cubes for which the Parrondo effect is 
present at the center point. The simulation estimate p is also based on n = 10 6 
points but here they are randomly chosen. The standard error is the square root of 
p{l — p)/n. (Assumptions: See the caption to Figure [I]) 



N 


integrated 


Riemann sum 


simulation 


standard 




volume 


approximation 


estimate 


error of 






to volume 


of volume 


simulation 


3 


0.0174361 


0.017314 
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0.0001305 
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0.029199 


0.029227 


0.0001684 


5 
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0.0001067 


6 
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7 
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10 
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11 




0.006491 
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0.0000821 


12 




0.006356 


0.006622 


0.0000811 


13 




0.006227 


0.006492 


0.0000803 



Table 3: Analysis of the Parrondo effect for Toral's choice of the probability parame- 
ters. Ellipses are intended to suggest that exact numbers have been truncated at six 
digits. The last two columns are rounded to six significant digits. (Assumptions: See 
the caption to Figure [l]) 
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Table 4: Analysis of the Parrondo effect for a second point on the boundary of the 
unit cube and for a point in the interior. (Assumptions: See the caption to Figure nl) 
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domly mixed game C :— (1/2) (A + B). The model is described by a parameterized 
Markov chain with 2^ states. To maximize the value of N for which exact computa- 
tions are feasible, we regarded states as equivalent if they are equal after a rotation 
and/or reflection of the players. This allowed us to compute the mean profits per turn, 
[iB and fJLc-i to the ensemble of N players for 3 < N < 19 and several choices of the 
parameter vector (j)q,Pi,P3), including Toral's choice. The results provide numerical 
evidence, but not a proof, that [Ib and fic converge as N — > oo and that the Parrondo 
effect (i.e., \ib < and nc > 0) persists for all N sufficiently large for a set of pa- 
rameter vectors having nonzero volume. This suggests that the spatially-dependent 
version of Parrondo's paradox is a robust phenomenon that remains present in the 
thermodynamic limit. 

This is the main conclusion, but there are several other noteworthy conclusions. 
We have shown that the sequence of profits to the ensemble of N players obeys the 
strong law of large numbers. This is important in defining what is meant by a winning, 
losing, and fair game. For 3 < N < 6 explicit formulas for [i B and \iq are available, so 
with the help of computer graphics, we have demonstrated that one can visualize the 
Parrondo region, the region in the three-dimensional parameter space in which the 
Parrondo effect appears. There is also an anti-Parrondo region, and we have shown 
that it is symmetric with the Parrondo region, as might be expected. Finally, we have 
pointed out a close relationship between the spatially dependent Parrondo games in 
the case of N = 3 players and the one-player history-dependent Parrondo games. 

We have restricted our attention to the one-dimensional version of the model, but 
our methods may have applicability to the two-dimensional version, already investi- 
gated by Mihailovic and Rajkovic [12] using computer simulation. There are also, of 
course, several problems for mathematicians, including a proof that \ib and jic ac- 
tually do converge as N — > oo. We have partial results in this direction, which relate 
the Markov chain model to an interacting particle system, or spin system (Liggett 
[17] . Chapter 3). Spin systems are continuous-time Markov processes in the infinite- 
product space {0, 1} S , where S is a countable set such as the d-dimensional integer 
lattice. Some of the best-known spin systems (e.g., the stochastic Ising model) were 
motivated by physics. It is interesting that two mathematical models (Parrondo 
games and spin systems), with rather different physical motivations, should come 
together in this way. 
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n = 10; a = Cycles [{{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}}]; (* cyclic permutation *) 

b = Cycles[{{l, 10}, {2, 9}, {3, 8}, {4, 7}, {5, 6}}]; (* order-reversing permutation *) 

group = PermutationGroup[{a, b} ] ; (* dihedral group *) 

sigma = IntegerDigits [Range [0, 2'n - 1] , 2, n] ; 

Do [permuted [ i] = Permute [sigma [[ i] ] , group], {i, 1, 2*n}]; 

(* orbit of ith element of Sigma under G, with duplication *) 

Do[digit[x_] := FromDigits [permuted [i] [ [x] ] , 2]; list[i] = Table [digit [x] , {x, 1, 2 n} ] , {i, 1, 2"n}]; 
(* set of orbit elements in decimal form, with duplication *) 
class[l] = {0}; (* the first equivalence class *) num = 1; 
For[j = 2, j 5 2- (n - 1) , 
For[test = 1 ; k = 1 , k 5 j - 1 , k+ + . If [Sort [list [k] ] == Sort[list[j] ] , test = 0] ] ; 
If[test==l, num = num + 1 ; class [num] = DeleteDuplicates [Sort [list [j ]]]]] ; 
( * generates list of equivalence classes * ) 
num = num + 1; (* number of equivalence classes *) 
class [num] = {2 A n - 1}; (* the last equivalence class *) 

For[i = l, is num, i++, state[i] = IntegerDigits [class [ i] , 2, n] ] ; 
(* the binary states belonging to equivalence class i *) 
For [ i = 1 , i < num , i + + , 

ones [i] = (sum = 0; For [k =1, kin, k+ + . If [state [i] [ [1, k] ] ==1, sum += 1] ] ; sum) ] ; 
(* number of ones in each element of equivalence class i *) 

diff [x_, y_] := (sum = 0; For[k =1, k <, n, k+ + , sum += Abs[x[ [k] ] - y[ [k] ] ] ] ; sum) ; 
(* the Hamming distance between states x and y *) 

p[0] =p0; p[l] =pl; p[2] =p2; p[3] =p3; q[0] = qO; q[l] = ql ; q[2] = q2 ; q[3] = q3 ; 

pbar - ConstantArray [ , {num, num}]; 

For [i = 1 , i 5 num, i + + , For [ j = 1 , j £ num, j + + , 

(* evaluates Pbar based on equation (8) *)If[i==j, For[k = l, k< n, k+ + , 

If [state[i] [ [1, k] ] == 1, If[k=l, Pbar[[i, i] ] + = p [2 state [ i] [ [ 1 , n] ] * state [i] [ [ 1 , 2] ] ] / n , 
If[k==n, Pbar[[i, i] ] + = p [ 2 state [ i] [ [ 1 , n - 1] ] * state [i] [[ 1 , 1] ]]/ n, 
Pbar[[i, i] ] + = p [2 state [ i] [ [ 1 , k - 1] ] * state [i] [[ 1 , k + 1] ]]/ n] ] , 
If[k==l, Pbar[[i, i] ] + = q [2 state [ i] [ [ 1 , n] ] * state [i] [ [ 1 , 2] ] ] / n , 
If[k==n, Pbar[[i, i] ] + = q[ 2 state [ i] [ [ 1 , n - 1] ] * state [i] [[ 1 , 1] ]]/ n, 
Pbar[[i, i] ] + = q [2 state [ i] [ [ 1 , k - 1] ] * state [i] [[ 1 , k + 1] ]]/ n] ]]] , 
If [ones [ j ] == ones [ i] + 1 , For [m = 1, m £ Length [class [ j ] ] , m + + , 

If [diff [state[ j] [[m]] , state [ i] [[ 1] ] ] ==1, For[k=l, ks n, k + t, If [ state [ j ][ [m, k 

]] > state[i] [ [1, k] ] , If [k == 1, Pbar[[i, j]] + = p [2 state [i] [ [ 1 , n] ] + state [ i] [[ 1 , 

2]]]/n, If[k==n, Pbar[[i, j]] + = p [ 2 state [ i] [ [ 1 , n - 1] ] + state [i] [[ 1 , 1] ]]/ n, 
Pbar[[i, j]] + = p[2state[i] [[1, k- 1] ] + state [ i] [[ 1 , k * 1] ]]/ n] ]]]]] , 
If [ones [ j ] == ones [i] -1, For [m - 1 , m < Length [class [ j ] ] , ra + + , 

If [dif f [ state [ j] [ [m] ] , state [i] [ [1] ] ] ==1, For[k = 1, k s n, k++, If [ state [ j] [ [m, k 

]] < state[i] [ [1, k] ] , If [k == 1, Pbar[[i, j]] + = q[ 2 state [ i] [ [ 1 , n] ] + state [i] [[ 1 , 

2]]]/n, If[k==n, Pbar[[i, j]] += q[ 2 state [ i] [ [ 1 , n - 1] ] + state [ i] [[ 1 , 1] ]]/ n, 
Pbar[[i, j]] + = q[2state[i] [[1, k - 1] ] + state[i] [[1, k * 1] ]]/ n] ]]]]]]]]]] ; 

Pbardot = ConstantArray [0, {num, num}]; 

(* Insert here the 18-line For statement that follows "Pbar=ConstantArray [0 , {num, num} ] ; " , 
but with two changes: replace "Pbar" by "Pbardot" and "+= q" by "-= q" throughout. *) 
one = ConstantArray [ 1 , {num, 1}]; Pbardotone = Pbardot . one; 

pO = 1; pi = 4/25; p2 = pi; p3 = 7/10; qO = 1 - pO; ql = 1 - pi ; q2 = 1 - p2; q3 = 1 - p3 ; 
pibar = Array [x, {num}]; sol = Solve [ {pibar == pibar . Pbar , pibar . one -- 1 } , pibar] ; 
muB = pibar. Pbardotone /. sol 

pO = (1/ 2 + 1) / 2; pi = (1/2 + 4/25) 12; p2 = pi; p3 = ( 1 / 2 + 7 / 10) / 2 ; 

qO = 1 - pO; ql = 1 - pi; q2 = 1 - p2; q3 = 1 - p3; pibar = Array [x, {num}]; 

sol = Solve[{pibar == pibar. Pbar, pibar . one -- 1 } , pibar] ; muC = pibar . Pbardotone /. sol 



Figure 3: The Mathematica program used to enumerate the equivalence classes, com- 
pute P and PI, solve for the stationary distribution tt, and evaluate hb and /ic- 
Here N — 10 and pi = p 2 - But N can be changed by modifying the first three 
instructions, and the assumption pi — p 2 can be dropped by eliminating b in line 3 
and changing 2n to n in line 7. 
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